Virtualization software deployments are allowing organizations to achieve significant savings in their data centers. These savings are being gained in reduced energy and hardware costs, as well as increasing the flexibility they have in the deployment of their mission-critical applications. Deployments are often seeing 10 or more virtual machines deployed on a single physical server. The driver for virtualization deployments was initially consolidation of resources, however the benefits achieved are now causing virtualization software to fundamentally affect how mission-critical applications are designed, deployed and managed. Virtualization deployments are also causing organizations to consider which security mechanisms can improve the security posture of their physical and virtual server systems.
The ability for malware to remotely exploit software vulnerabilities is the primary threat to virtualized environments as it is with physical servers.
FIG. 1 illustrates a first virtual server system 100 of the prior art, including a Hypervisor 110, connected to one or more virtual switches (v-Switch) 120, each vSwitch 120 connected to one or more Guest Virtual Machines (Guest VM) 130. The Hypervisor 110 generally supports a Console Operating System (Console OS) 140. Each Guest VM 130 comprises an Operating System (OS) 150, and may include one or more Enterprise Applications 160 and one or more Web Applications 170.
In the first virtual server system 100, the Guest VMs 130 have access to computing resources such as networking resources only through the Hypervisor 110. The function of each v-Switch 120 is to isolate multiple Guest VMs 130 from each other while giving each Guest VM 130 access to the Hypervisor 110.
The primary location of the vulnerabilities are in the OS 150, the enterprise software (the Enterprise Applications 160), and the custom applications (the Web Applications 170) that may exist on the Guest VMs 130 as illustrated in FIG. 1. Other vulnerabilities may exist in the service console software (the Console OS 140) and the Hypervisor 110.
The service console is an asset which can have remotely exploitable vulnerabilities. Virtualization vendors such as Vmware, Inc. of Palo Alto, Calif., continue to work to simplify the service console. A white paper on VMware Infrastructure entitled “VMware Infrastructure Architecture Overview” was published by VMware on Jun. 5, 2006, which cited in the Information Disclosure Statement submitted by the applicant. Hypervisor vulnerabilities are not typically remotely exploitable, since the hypervisor does not have services which terminate protocols that could lead to input validation errors. Hypervisor vulnerabilities will typically be exploited from malware, which gets on to a virtual machine, i.e. on of the Guest VMs 130 and one of the best methods to protect against this is to protect against the malware getting installed there in the first place.
In protecting software vulnerabilities, virtualized environments present similar challenges for the intrusion-detection systems and intrusion-prevention systems (IDS/IPS) typically deployed in a data center, but they also present some new challenges and opportunities. It is now clear that hardware appliance based security is not sufficient to protect virtualized environments, since these devices are blind to the traffic being sent between virtual machines on a physical server. In addition, the dynamic nature of virtualized environments, with older snapshots being quickly restored and virtual machines being moved between physical servers to optimize resource use present challenges that do not exist with physical servers. The opportunity presented is that the investment in multi-processor, multi-core computing and the virtualization layer that manages it can also be leveraged to deploy the security mechanisms required to secure it.
When deciding on an approach to virtualization security, organizations can use similar security principles that have emerged in their physical environments over the last few years. One of these principles is “defence-in-depth”, which is a fundamental security requirement for organizations that recognize the “de-perimeterization” that has emerged in their information technology deployments. The Jericho Forum has defined a set of commandments that should be observed when planning for a de-perimeterized future, published at “http://www.opengroup.org/jericho/commandments_v1.1.pdf”.
The first fundamental of the Jericho Forum commandments is:                1. The scope and level of protection should be specific and appropriate to the asset at risk:                    Business demands that security enables business agility and is cost effective;            Whereas boundary firewalls may continue to provide basic network protection, individual systems and data will need to be capable of protecting themselves;            In general, it's easier to protect an asset the closer protection is provided.                        
Virtualization has made the challenge of de-perimeterization even more challenging. The inability of appliance based security to deal with attacks between virtual machines on the same server makes clear the need for mechanisms to be deployed on the server to protect these environments. A requirement has emerged for a virtualization security approach that allows protection of an asset to occur as close as possible to the asset itself.
There are two approaches currently being taken to protecting virtual machines with security software on the server.
FIG. 2A illustrates a second virtual server system 200 of the prior art, in which a security overlay is provided within the virtualized environment. The second virtual server system 200 differs from the first virtual server system 100 in that IDS/IPS security appliances 220 are added within the virtualized environment and connected to the one or more Guest VMs 130 through the vSwitches 120. The IDS/IPS security appliances 220 are connected to the hypervisor 110 through additional vSwitches 240. Each of the security appliances 220 monitor the traffic flow between the respective additional vSwitch 240 and the respective vSwitch 120 and thus provide security to the Guest VMs 130 that are connected to the respective vSwitch 120.
Such security overlay solutions provide protection for Intrusion Detection System (IDS)/Intrusion Prevention System (IPS) against attacks which are coming from the network, but they have significant limitations. These limitations include:                Inter-VM traffic. The virtual appliance must be placed in front of a v-Switch 120 and therefore can not prevent attacks between virtual machines on the same v-Switch 120.        Mobility. When a virtual machine is moved from one server to another using controls like Vmotion from VMware, the security context is lost. Clustering of the security appliances must be deployed in advance to all potential destinations to which a virtual machine could be moved, with the corresponding negative performance impacts.        Non-transparent. The virtual network architecture (v-Switch) must be altered to deploy virtual appliances such as the IDS/IPS security appliances 220. This has administrative and performance impacts on the existing system.        Performance bottleneck. All traffic which is sent between virtual machines and the network must be processed by the overlay IDS/IPS. This is can be a centralized performance bottleneck, having significant CPU impact and latency impact on the data flow.        
Accordingly, there is a need in the industry for further development of and improved method and system for intrusion prevention/detection in a virtualized environment, which would avoid or mitigate shortcomings of the prior art.